additional comment
OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking Supplementary Material
Motivation For what purpose was the dataset created? Was there a specific task in mind? Was there a specific gap that needed to be filled? In the current task of open-vocabulary multi-object tracking (OVMOT), there is only one benchmark available, which lacks high-quality, large-scale datasets. The existing dataset suffers from several limitations, including insufficient categories, limited video data, and a significant imbalance between base classes and novel classes. These deficiencies make it inadequate for supporting the evaluation of new OVMOT models. Our proposed dataset aims to provide a more comprehensive evaluation platform for the OVMOT task. Who created this dataset (e.g., which team, research group) and on behalf of which entity (e.g., company, institution, organization)? This dataset was constructed by collecting and extracting data from seven other datasets and applying unified annotations. This work was completed by Haiji Liang and Ruize Han. Who funded the creation of the dataset?
What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialects
Blaschke, Verena, Purschke, Christoph, Schรผtze, Hinrich, Plank, Barbara
Natural language processing (NLP) has largely focused on modelling standardized languages. More recently, attention has increasingly shifted to local, non-standardized languages and dialects. However, the relevant speaker populations' needs and wishes with respect to NLP tools are largely unknown. In this paper, we focus on dialects and regional languages related to German -- a group of varieties that is heterogeneous in terms of prestige and standardization. We survey speakers of these varieties (N=327) and present their opinions on hypothetical language technologies for their dialects. Although attitudes vary among subgroups of our respondents, we find that respondents are especially in favour of potential NLP tools that work with dialectal input (especially audio input) such as virtual assistants, and less so for applications that produce dialectal output such as machine translation or spellcheckers.
Eliciting Informative Text Evaluations with Large Language Models
Lu, Yuxuan, Xu, Shengwei, Zhang, Yichi, Kong, Yuqing, Schoenebeck, Grant
Peer prediction mechanisms motivate high-quality feedback with provable guarantees. However, current methods only apply to rather simple reports, like multiple-choice or scalar numbers. We aim to broaden these techniques to the larger domain of text-based reports, drawing on the recent developments in large language models. This vastly increases the applicability of peer prediction mechanisms as textual feedback is the norm in a large variety of feedback channels: peer reviews, e-commerce customer reviews, and comments on social media. We introduce two mechanisms, the Generative Peer Prediction Mechanism (GPPM) and the Generative Synopsis Peer Prediction Mechanism (GSPPM). These mechanisms utilize LLMs as predictors, mapping from one agent's report to a prediction of her peer's report. Theoretically, we show that when the LLM prediction is sufficiently accurate, our mechanisms can incentivize high effort and truth-telling as an (approximate) Bayesian Nash equilibrium. Empirically, we confirm the efficacy of our mechanisms through experiments conducted on two real datasets: the Yelp review dataset and the ICLR OpenReview dataset. We highlight the results that on the ICLR dataset, our mechanisms can differentiate three quality levels -- human-written reviews, GPT-4-generated reviews, and GPT-3.5-generated reviews in terms of expected scores. Additionally, GSPPM penalizes LLM-generated reviews more effectively than GPPM.
Training Classes - Data, Artificial Intelligence & Advanced Analytics Summit
Training Classes are 8 hours, focused, deep-dive, demo-based virtual classroom training. Each class will run for eight hours in total, four hours each day, for two consecutive days. There will be three batches spread across three different time zones. Each batch will have four parallel classes. Click Here to learn about our global delivery model. Each Training Class is designed to offer intermediate & advanced-level training on a specific topic/subject. These classes offer more knowledge, skills, and expertise beyond the summit content. Apart from attending LIVE classes, you also get on-demand access to class recordings*, exclusive content from the instructor, and the participation certificate.